Image Captioning Through Image Transformer

Automatic captioning of images is a task that combines the challenges image analysis and text generation. One important aspect notion attention: how to decide what describe in which order. Inspired by successes translation, previous works have proposed transformer architecture for captioning. However, structure between semantic units (usually detected regions from object detection model) sentences (each single word) different. Limited work has been done adapt transformer’s internal images. In this work, we introduce transformer, consists modified encoding an implicit decoding motivated relative spatial relationship regions. Our design widens original layer’s inner With only feature as inputs, our model achieves new state-of-the-art performance on both MSCOCO offline online testing benchmarks. The code available at https://github.com/wtliao/ImageTransformer.